<!DOCTYPE html>
<html lang="en-US">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <title>《统计学习方法》第 2 章“感知机”学习笔记 | 算法不好玩</title>
    <meta name="generator" content="VuePress 1.8.2">
    <link rel="icon" href="/book/logo.png">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.7.1/katex.min.css">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/2.10.0/github-markdown.min.css">
    <meta name="description" content="">
    
    <link rel="preload" href="/book/assets/css/0.styles.e159a327.css" as="style"><link rel="preload" href="/book/assets/js/app.a4355b0d.js" as="script"><link rel="preload" href="/book/assets/js/2.ba785ee6.js" as="script"><link rel="preload" href="/book/assets/js/46.33bc2083.js" as="script"><link rel="prefetch" href="/book/assets/js/10.b6950be1.js"><link rel="prefetch" href="/book/assets/js/11.f4ee7d6e.js"><link rel="prefetch" href="/book/assets/js/12.50fe1ace.js"><link rel="prefetch" href="/book/assets/js/13.42356fcc.js"><link rel="prefetch" href="/book/assets/js/14.e740c742.js"><link rel="prefetch" href="/book/assets/js/15.05c628f7.js"><link rel="prefetch" href="/book/assets/js/16.bd9bc9d5.js"><link rel="prefetch" href="/book/assets/js/17.e120b4fc.js"><link rel="prefetch" href="/book/assets/js/18.398213f8.js"><link rel="prefetch" href="/book/assets/js/19.e29ad0b2.js"><link rel="prefetch" href="/book/assets/js/20.14e30c1c.js"><link rel="prefetch" href="/book/assets/js/21.4f05f6c9.js"><link rel="prefetch" href="/book/assets/js/22.98fbf199.js"><link rel="prefetch" href="/book/assets/js/23.285387f4.js"><link rel="prefetch" href="/book/assets/js/24.852addbe.js"><link rel="prefetch" href="/book/assets/js/25.d30ade13.js"><link rel="prefetch" href="/book/assets/js/26.23dfa040.js"><link rel="prefetch" href="/book/assets/js/27.b6eae7df.js"><link rel="prefetch" href="/book/assets/js/28.97878b08.js"><link rel="prefetch" href="/book/assets/js/29.7217a3d0.js"><link rel="prefetch" href="/book/assets/js/3.f0c15194.js"><link rel="prefetch" href="/book/assets/js/30.0ced5a5a.js"><link rel="prefetch" href="/book/assets/js/31.8f432033.js"><link rel="prefetch" href="/book/assets/js/32.aac9aa62.js"><link rel="prefetch" href="/book/assets/js/33.db835477.js"><link rel="prefetch" href="/book/assets/js/34.a1a59c2a.js"><link rel="prefetch" href="/book/assets/js/35.ed2ef96b.js"><link rel="prefetch" href="/book/assets/js/36.48099c55.js"><link rel="prefetch" href="/book/assets/js/37.32827612.js"><link rel="prefetch" href="/book/assets/js/38.58abec0e.js"><link rel="prefetch" href="/book/assets/js/39.f6eb3874.js"><link rel="prefetch" href="/book/assets/js/4.13ea3b32.js"><link rel="prefetch" href="/book/assets/js/40.80523ed8.js"><link rel="prefetch" href="/book/assets/js/41.f7b72b7a.js"><link rel="prefetch" href="/book/assets/js/42.44e40de9.js"><link rel="prefetch" href="/book/assets/js/43.065f077f.js"><link rel="prefetch" href="/book/assets/js/44.386d9aea.js"><link rel="prefetch" href="/book/assets/js/45.d8a88f16.js"><link rel="prefetch" href="/book/assets/js/47.e7767237.js"><link rel="prefetch" href="/book/assets/js/48.9bb76138.js"><link rel="prefetch" href="/book/assets/js/49.c2ca5c29.js"><link rel="prefetch" href="/book/assets/js/5.32eda204.js"><link rel="prefetch" href="/book/assets/js/50.1242fe24.js"><link rel="prefetch" href="/book/assets/js/51.e8f8117d.js"><link rel="prefetch" href="/book/assets/js/52.eae5648f.js"><link rel="prefetch" href="/book/assets/js/53.39876d83.js"><link rel="prefetch" href="/book/assets/js/6.1c662163.js"><link rel="prefetch" href="/book/assets/js/7.29470445.js"><link rel="prefetch" href="/book/assets/js/8.814b737f.js"><link rel="prefetch" href="/book/assets/js/9.d4211262.js">
    <link rel="stylesheet" href="/book/assets/css/0.styles.e159a327.css">
  </head>
  <body>
    <div id="app" data-server-rendered="true"><div class="theme-container no-sidebar"><header class="navbar"><div class="sidebar-button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" role="img" viewBox="0 0 448 512" class="icon"><path fill="currentColor" d="M436 124H12c-6.627 0-12-5.373-12-12V80c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12z"></path></svg></div> <a href="/book/" class="home-link router-link-active"><!----> <span class="site-name">算法不好玩</span></a> <div class="links"><div class="search-box"><input aria-label="Search" autocomplete="off" spellcheck="false" value=""> <!----></div> <nav class="nav-links can-hide"><div class="nav-item"><a href="/book/" class="nav-link">
  主页
</a></div><div class="nav-item"><a href="/book/guide/" class="nav-link">
  关于我
</a></div><div class="nav-item"><div class="dropdown-wrapper"><button type="button" aria-label="基础算法与数据结构" class="dropdown-title"><span class="title">基础算法与数据结构</span> <span class="arrow down"></span></button> <button type="button" aria-label="基础算法与数据结构" class="mobile-dropdown-title"><span class="title">基础算法与数据结构</span> <span class="arrow right"></span></button> <ul class="nav-dropdown" style="display:none;"><li class="dropdown-item"><h4>
          排序算法
        </h4> <ul class="dropdown-subitem-wrapper"><li class="dropdown-subitem"><a href="/book/algs/01-binary-search/" class="nav-link">
  第 1 章 二分查找
</a></li><li class="dropdown-subitem"><a href="/book/algs/recursion/" class="nav-link">
  第 2 章 递归
</a></li></ul></li><li class="dropdown-item"><h4>
          数组里的算法
        </h4> <ul class="dropdown-subitem-wrapper"><li class="dropdown-subitem"><a href="/book/algs/01-binary-search/" class="nav-link">
  第 4 章 滑动窗口
</a></li><li class="dropdown-subitem"><a href="/book/algs/02-basic-sorting/" class="nav-link">
  第 5 章 双指针
</a></li></ul></li></ul></div></div><div class="nav-item"><div class="dropdown-wrapper"><button type="button" aria-label="威威道来" class="dropdown-title"><span class="title">威威道来</span> <span class="arrow down"></span></button> <button type="button" aria-label="威威道来" class="mobile-dropdown-title"><span class="title">威威道来</span> <span class="arrow right"></span></button> <ul class="nav-dropdown" style="display:none;"><li class="dropdown-item"><!----> <a href="/book/talk-show/2021-04/" class="nav-link">
  2021 年 4 月
</a></li></ul></div></div><div class="nav-item"><a href="https://leetcode-cn.com/" target="_blank" rel="noopener noreferrer" class="nav-link external">
  力扣
  <span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></div><div class="nav-item"><a href="https://gitee.com/liweiwei1419/book/pages" target="_blank" rel="noopener noreferrer" class="nav-link external">
  Gitee 部署页面
  <span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></div> <!----></nav></div></header> <div class="sidebar-mask"></div> <aside class="sidebar"><nav class="nav-links"><div class="nav-item"><a href="/book/" class="nav-link">
  主页
</a></div><div class="nav-item"><a href="/book/guide/" class="nav-link">
  关于我
</a></div><div class="nav-item"><div class="dropdown-wrapper"><button type="button" aria-label="基础算法与数据结构" class="dropdown-title"><span class="title">基础算法与数据结构</span> <span class="arrow down"></span></button> <button type="button" aria-label="基础算法与数据结构" class="mobile-dropdown-title"><span class="title">基础算法与数据结构</span> <span class="arrow right"></span></button> <ul class="nav-dropdown" style="display:none;"><li class="dropdown-item"><h4>
          排序算法
        </h4> <ul class="dropdown-subitem-wrapper"><li class="dropdown-subitem"><a href="/book/algs/01-binary-search/" class="nav-link">
  第 1 章 二分查找
</a></li><li class="dropdown-subitem"><a href="/book/algs/recursion/" class="nav-link">
  第 2 章 递归
</a></li></ul></li><li class="dropdown-item"><h4>
          数组里的算法
        </h4> <ul class="dropdown-subitem-wrapper"><li class="dropdown-subitem"><a href="/book/algs/01-binary-search/" class="nav-link">
  第 4 章 滑动窗口
</a></li><li class="dropdown-subitem"><a href="/book/algs/02-basic-sorting/" class="nav-link">
  第 5 章 双指针
</a></li></ul></li></ul></div></div><div class="nav-item"><div class="dropdown-wrapper"><button type="button" aria-label="威威道来" class="dropdown-title"><span class="title">威威道来</span> <span class="arrow down"></span></button> <button type="button" aria-label="威威道来" class="mobile-dropdown-title"><span class="title">威威道来</span> <span class="arrow right"></span></button> <ul class="nav-dropdown" style="display:none;"><li class="dropdown-item"><!----> <a href="/book/talk-show/2021-04/" class="nav-link">
  2021 年 4 月
</a></li></ul></div></div><div class="nav-item"><a href="https://leetcode-cn.com/" target="_blank" rel="noopener noreferrer" class="nav-link external">
  力扣
  <span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></div><div class="nav-item"><a href="https://gitee.com/liweiwei1419/book/pages" target="_blank" rel="noopener noreferrer" class="nav-link external">
  Gitee 部署页面
  <span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></div> <!----></nav>  <!----> </aside> <main class="page"> <div class="theme-default-content content__default"><div class="custom-block tip"><p class="custom-block-title">TIP</p> <p>本文介绍了感知机算法。</p></div> <p><img src="https://ws1.sinaimg.cn/large/006tKfTcly1g0tzql2kilj32ee0rmti1.jpg" alt="感知机模型"></p> <p>感知机是《统计学习方法》的介绍的第 1 个算法，是神经网络与 SVM 的基础。</p> <h3 id="研究思路"><a href="#研究思路" class="header-anchor">#</a> 研究思路</h3> <p>1、模型：二分类问题，数据点分为“<mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mo class="mjx-n"><mjx-c c="+"></mjx-c></mjx-mo><mjx-mn class="mjx-n"><mjx-c c="1"></mjx-c></mjx-mn></mjx-math></mjx-container>”类和“<mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mo class="mjx-n"><mjx-c c="2212"></mjx-c></mjx-mo><mjx-mn class="mjx-n"><mjx-c c="1"></mjx-c></mjx-mn></mjx-math></mjx-container>”类，“超平面”为所求；</p> <p>2、策略：损失函数最小化，确定参数 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="w"></mjx-c></mjx-mi></mjx-math></mjx-container> 和 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="b"></mjx-c></mjx-mi></mjx-math></mjx-container>；</p> <p>3、算法：随机梯度下降法。</p> <h3 id="策略-随机梯度下降"><a href="#策略-随机梯度下降" class="header-anchor">#</a> 策略：随机梯度下降</h3> <p>用普通的基于所有样本的梯度和的均值的批量梯度下降法（BGD）是行不通的，原因在于我们的损失函数里面有限定，<strong>只有误分类的 M 集合里面的样本才能参与损失函数的优化</strong>。所以我们不能用最普通的批量梯度下降,只能采用随机梯度下降（SGD）或者小批量梯度下降（MBGD）。</p> <h3 id="感知机学习的对偶形式"><a href="#感知机学习的对偶形式" class="header-anchor">#</a> 感知机学习的对偶形式</h3> <p>将 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="w"></mjx-c></mjx-mi></mjx-math></mjx-container> 和 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="b"></mjx-c></mjx-mi></mjx-math></mjx-container> 表示成实例 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-msub><mjx-mi noIC="true" class="mjx-i"><mjx-c c="x"></mjx-c></mjx-mi><mjx-script style="vertical-align:-0.15em;"><mjx-mi size="s" class="mjx-i"><mjx-c c="i"></mjx-c></mjx-mi></mjx-script></mjx-msub></mjx-math></mjx-container> 和 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-msub><mjx-mi noIC="true" class="mjx-i"><mjx-c c="y"></mjx-c></mjx-mi><mjx-script style="vertical-align:-0.15em;"><mjx-mi size="s" class="mjx-i"><mjx-c c="i"></mjx-c></mjx-mi></mjx-script></mjx-msub></mjx-math></mjx-container> 的线性组合。</p> <p>1、<mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-msub><mjx-mi noIC="true" class="mjx-i"><mjx-c c="w"></mjx-c></mjx-mi><mjx-script style="vertical-align:-0.15em;"><mjx-mn size="s" class="mjx-n"><mjx-c c="0"></mjx-c></mjx-mn></mjx-script></mjx-msub><mjx-mo space="4" class="mjx-n"><mjx-c c="="></mjx-c></mjx-mo><mjx-TeXAtom space="4"><mjx-mover><mjx-over style="padding-bottom:0.06em;margin-bottom:-0.516em;"><mjx-mo class="mjx-n"><mjx-c c="20D7"></mjx-c></mjx-mo></mjx-over><mjx-base><mjx-mn class="mjx-n"><mjx-c c="0"></mjx-c></mjx-mn></mjx-base></mjx-mover></mjx-TeXAtom></mjx-math></mjx-container> ，<mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-msub><mjx-mi noIC="true" class="mjx-i"><mjx-c c="b"></mjx-c></mjx-mi><mjx-script style="vertical-align:-0.15em;"><mjx-mn size="s" class="mjx-n"><mjx-c c="0"></mjx-c></mjx-mn></mjx-script></mjx-msub><mjx-mo space="4" class="mjx-n"><mjx-c c="="></mjx-c></mjx-mo><mjx-mn space="4" class="mjx-n"><mjx-c c="0"></mjx-c></mjx-mn></mjx-math></mjx-container>，则 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="w"></mjx-c></mjx-mi></mjx-math></mjx-container> 和 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="b"></mjx-c></mjx-mi></mjx-math></mjx-container> 就可以表示成 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-msub><mjx-mi noIC="true" class="mjx-i"><mjx-c c="x"></mjx-c></mjx-mi><mjx-script style="vertical-align:-0.15em;"><mjx-mi size="s" class="mjx-i"><mjx-c c="i"></mjx-c></mjx-mi></mjx-script></mjx-msub></mjx-math></mjx-container> 和 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-msub><mjx-mi noIC="true" class="mjx-i"><mjx-c c="y"></mjx-c></mjx-mi><mjx-script style="vertical-align:-0.15em;"><mjx-mi size="s" class="mjx-i"><mjx-c c="i"></mjx-c></mjx-mi></mjx-script></mjx-msub></mjx-math></mjx-container> 的线性组合；</p> <p>2、实现技巧：向量化代替 for 循环。</p> <h3 id="对于损失函数的理解"><a href="#对于损失函数的理解" class="header-anchor">#</a> 对于损失函数的理解</h3> <p>感知机学习固定分母为 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mn class="mjx-n"><mjx-c c="1"></mjx-c></mjx-mn></mjx-math></mjx-container>。我们研究可以发现，分子和分母都含有 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="w"></mjx-c></mjx-mi></mjx-math></mjx-container>，当分子的 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="w"></mjx-c></mjx-mi></mjx-math></mjx-container> 扩大 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="N"></mjx-c></mjx-mi></mjx-math></mjx-container> 倍时，分母的 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-msub><mjx-mi noIC="true" class="mjx-i"><mjx-c c="L"></mjx-c></mjx-mi><mjx-script style="vertical-align:-0.15em;"><mjx-mn size="s" class="mjx-n"><mjx-c c="2"></mjx-c></mjx-mn></mjx-script></mjx-msub></mjx-math></mjx-container> 范数也会扩大 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mi class="mjx-i"><mjx-c c="N"></mjx-c></mjx-mi></mjx-math></mjx-container> 倍。也就是说，分子和分母有固定的倍数关系。那么我们可以固定分子或者分母为 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mn class="mjx-n"><mjx-c c="1"></mjx-c></mjx-mn></mjx-math></mjx-container>，然后求另一个即分子自己或者分母的倒数的最小化作为损失函数，这样可以简化我们的损失函数。在感知机模型中，我们采用的是保留分子，即最终感知机模型的损失函数简化为：</p> <p><span class="katex-display"><span class="katex"><span class="katex-mathml"><math><semantics><mrow><mi>J</mi><mo>(</mo><mi>θ</mi><mo>)</mo><mo>=</mo><mo>−</mo><msub><mo>∑</mo><mrow><msub><mi>x</mi><mi>i</mi></msub><mo>∈</mo><mi>M</mi></mrow></msub><msup><mi>y</mi><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow></msup><mi>θ</mi><mo>⋅</mo><msup><mi>x</mi><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow></msup></mrow><annotation encoding="application/x-tex">J(\theta) = - \sum\limits_{x_i \in M}y^{(i)}\theta \cdot x^{(i)}
</annotation></semantics></math></span><span aria-hidden="true" class="katex-html"><span class="strut" style="height:1.0500050000000003em;"></span><span class="strut bottom" style="height:2.4493410000000004em;vertical-align:-1.399336em;"></span><span class="base displaystyle textstyle uncramped"><span class="mord mathit" style="margin-right:0.09618em;">J</span><span class="mopen">(</span><span class="mord mathit" style="margin-right:0.02778em;">θ</span><span class="mclose">)</span><span class="mrel">=</span><span class="mord">−</span><span class="mop op-limits"><span class="vlist"><span style="top:1.1943359999999998em;margin-left:0em;"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span><span class="reset-textstyle scriptstyle cramped"><span class="mord scriptstyle cramped"><span class="mord"><span class="mord mathit">x</span><span class="vlist"><span style="top:0.15em;margin-right:0.07142857142857144em;margin-left:0em;"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span><span class="reset-scriptstyle scriptscriptstyle cramped"><span class="mord mathit">i</span></span></span><span class="baseline-fix"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span>​</span></span></span><span class="mrel">∈</span><span class="mord mathit" style="margin-right:0.10903em;">M</span></span></span></span><span style="top:-0.000005000000000143778em;"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span><span><span class="op-symbol large-op mop">∑</span></span></span><span class="baseline-fix"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span>​</span></span></span><span class="mord"><span class="mord mathit" style="margin-right:0.03588em;">y</span><span class="vlist"><span style="top:-0.413em;margin-right:0.05em;"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span><span class="reset-textstyle scriptstyle uncramped"><span class="mord scriptstyle uncramped"><span class="mopen">(</span><span class="mord mathit">i</span><span class="mclose">)</span></span></span></span><span class="baseline-fix"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span>​</span></span></span><span class="mord mathit" style="margin-right:0.02778em;">θ</span><span class="mbin">⋅</span><span class="mord"><span class="mord mathit">x</span><span class="vlist"><span style="top:-0.413em;margin-right:0.05em;"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span><span class="reset-textstyle scriptstyle uncramped"><span class="mord scriptstyle uncramped"><span class="mopen">(</span><span class="mord mathit">i</span><span class="mclose">)</span></span></span></span><span class="baseline-fix"><span class="fontsize-ensurer reset-size5 size5"><span style="font-size:0em;">​</span></span>​</span></span></span></span></span></span></span></p> <p>我的理解：反正最终都会收敛，所以损失函数收敛的时候一定为 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mn class="mjx-n"><mjx-c c="0"></mjx-c></mjx-mn></mjx-math></mjx-container>，因此分母是多少都无所谓，这就是书上说的“不考虑 <mjx-container jax="CHTML" class="MathJax"><mjx-math class=" MJX-TEX"><mjx-mfrac><mjx-frac><mjx-num><mjx-nstrut></mjx-nstrut><mjx-mrow size="s"><mjx-mpadded><mjx-block style="margin:0.896em 0 0.313em;"><mjx-mrow></mjx-mrow></mjx-block></mjx-mpadded><mjx-mstyle style="font-size:141.4%;"><mjx-TeXAtom><mjx-mn class="mjx-n"><mjx-c c="1"></mjx-c></mjx-mn></mjx-TeXAtom></mjx-mstyle></mjx-mrow></mjx-num><mjx-dbox><mjx-dtable><mjx-line></mjx-line><mjx-row><mjx-den><mjx-dstrut></mjx-dstrut><mjx-mrow size="s"><mjx-mpadded><mjx-block style="margin:0.896em 0 0.313em;"><mjx-mrow></mjx-mrow></mjx-block></mjx-mpadded><mjx-mstyle style="font-size:141.4%;"><mjx-TeXAtom><mjx-mo class="mjx-n"><mjx-c c="|"></mjx-c></mjx-mo><mjx-mo class="mjx-n"><mjx-c c="|"></mjx-c></mjx-mo><mjx-mi class="mjx-i"><mjx-c c="w"></mjx-c></mjx-mi><mjx-mo class="mjx-n"><mjx-c c="|"></mjx-c></mjx-mo><mjx-mo class="mjx-n"><mjx-c c="|"></mjx-c></mjx-mo></mjx-TeXAtom></mjx-mstyle></mjx-mrow></mjx-den></mjx-row></mjx-dtable></mjx-dbox></mjx-frac></mjx-mfrac></mjx-math></mjx-container>”。</p> <h3 id="手写笔记"><a href="#手写笔记" class="header-anchor">#</a> 手写笔记</h3> <p><img src="http://upload-images.jianshu.io/upload_images/414598-11c53ba0b2c6477f.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="&quot;感知机&quot;模型手写笔记-1"> <img src="http://upload-images.jianshu.io/upload_images/414598-e61557b5f7c972aa.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="&quot;感知机&quot;模型手写笔记-2"> <img src="http://upload-images.jianshu.io/upload_images/414598-7a2bcfc8ab9d7d09.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="&quot;感知机&quot;模型手写笔记-3"> <img src="http://upload-images.jianshu.io/upload_images/414598-9fc8f2c3fa325dae.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240" alt="&quot;感知机&quot;模型手写笔记-4"></p> <h3 id="编码实现"><a href="#编码实现" class="header-anchor">#</a> 编码实现</h3> <p>代码还可以在 <a href="https://github.com/liweiwei1419/Machine-Learning-is-Fun/blob/master/Perceptron-learning/playML/perceptron.py" target="_blank" rel="noopener noreferrer">这里<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 查看。</p> <p>Python 代码：</p> <div class="language-python line-numbers-mode"><pre class="language-python"><code><span class="token keyword">import</span> numpy <span class="token keyword">as</span> np


<span class="token keyword">class</span> <span class="token class-name">Perceptron</span><span class="token punctuation">:</span>
    <span class="token triple-quoted-string string">&quot;&quot;&quot;
    感知机分类器：假设数据集是线性可分的
    &quot;&quot;&quot;</span>

    <span class="token keyword">def</span> <span class="token function">__init__</span><span class="token punctuation">(</span>self<span class="token punctuation">,</span> eta<span class="token operator">=</span><span class="token number">0.01</span><span class="token punctuation">,</span> n_iter<span class="token operator">=</span><span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
        <span class="token triple-quoted-string string">&quot;&quot;&quot;

        :param eta: 学习率，between 0.0 and 1.0，float
        :param n_iter: 最大迭代次数，int
        &quot;&quot;&quot;</span>
        self<span class="token punctuation">.</span>eta <span class="token operator">=</span> eta
        self<span class="token punctuation">.</span>n_iter <span class="token operator">=</span> n_iter

    <span class="token keyword">def</span> <span class="token function">fit</span><span class="token punctuation">(</span>self<span class="token punctuation">,</span> X<span class="token punctuation">,</span> y<span class="token punctuation">)</span><span class="token punctuation">:</span>
        <span class="token comment"># 同李航《统计学习方法》P29</span>
        <span class="token comment"># &quot;1&quot; 表示偏置，即如果变量有 2 个，学习的权重就会有 3 个</span>
        <span class="token comment"># 感知机就是学习这一组参数向量</span>
        <span class="token comment"># 这里 y 只有两个取值，1 或者 -1</span>
        <span class="token comment"># target - self.predict(xi)，predict 函数返回 1 或者 -1</span>
        <span class="token comment"># 如果相同，则上式 = 0，即分类正确的点对权重更新没有帮助</span>
        self<span class="token punctuation">.</span>w_ <span class="token operator">=</span> np<span class="token punctuation">.</span>zeros<span class="token punctuation">(</span><span class="token number">1</span> <span class="token operator">+</span> X<span class="token punctuation">.</span>shape<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">)</span>

        <span class="token comment"># print(self.w_)</span>
        self<span class="token punctuation">.</span>errors_ <span class="token operator">=</span> <span class="token punctuation">[</span><span class="token punctuation">]</span>

        <span class="token keyword">for</span> _ <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span>self<span class="token punctuation">.</span>n_iter<span class="token punctuation">)</span><span class="token punctuation">:</span>
            <span class="token comment"># print('迭代次数', _)</span>
            <span class="token comment"># 表示这一轮分错的数据的个数</span>
            errors <span class="token operator">=</span> <span class="token number">0</span>
            <span class="token comment"># 把所有的数据都看一遍</span>
            <span class="token keyword">for</span> xi<span class="token punctuation">,</span> target <span class="token keyword">in</span> <span class="token builtin">zip</span><span class="token punctuation">(</span>X<span class="token punctuation">,</span> y<span class="token punctuation">)</span><span class="token punctuation">:</span>
                <span class="token comment"># 【注意】这个处理就包括了 target 和 self.predict 相等的情况，</span>
                <span class="token comment"># 如果相等，下面两行 self.w_[1:] 和 self.w_[0] 都不会更新</span>
                <span class="token comment"># 如果不等，相当于朝着父梯度方向走了一点点</span>
                <span class="token comment"># 随机梯度下降法，每次只使用一个数据更新权重</span>

                <span class="token comment"># print('实际',target,'预测',self.predict(xi))</span>
                <span class="token keyword">if</span> target <span class="token operator">==</span> self<span class="token punctuation">.</span>predict<span class="token punctuation">(</span>xi<span class="token punctuation">)</span><span class="token punctuation">:</span>
                    <span class="token keyword">continue</span>
                update <span class="token operator">=</span> self<span class="token punctuation">.</span>eta <span class="token operator">*</span> target
                <span class="token comment"># w</span>
                self<span class="token punctuation">.</span>w_<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">:</span><span class="token punctuation">]</span> <span class="token operator">+=</span> update <span class="token operator">*</span> xi
                <span class="token comment"># b</span>
                self<span class="token punctuation">.</span>w_<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span> <span class="token operator">+=</span> update

                errors <span class="token operator">+=</span> <span class="token builtin">int</span><span class="token punctuation">(</span>update <span class="token operator">!=</span> <span class="token number">0.0</span><span class="token punctuation">)</span>
            <span class="token comment"># 如果这一轮分类都正确，则感知机学习可以停止了</span>
            <span class="token keyword">if</span> errors <span class="token operator">==</span> <span class="token number">0</span><span class="token punctuation">:</span>
                <span class="token keyword">break</span>
            self<span class="token punctuation">.</span>errors_<span class="token punctuation">.</span>append<span class="token punctuation">(</span>errors<span class="token punctuation">)</span>
        <span class="token keyword">return</span> self

    <span class="token keyword">def</span> <span class="token function">net_input</span><span class="token punctuation">(</span>self<span class="token punctuation">,</span> X<span class="token punctuation">)</span><span class="token punctuation">:</span>
        <span class="token triple-quoted-string string">&quot;&quot;&quot;
        计算输出
        :param X:
        :return:
        &quot;&quot;&quot;</span>
        <span class="token comment"># X 是 m * n 型</span>
        <span class="token comment"># self.w_[1:] 是 n * 1 型，可以 dot</span>
        <span class="token comment"># + self.w_[0] 发生了广播</span>
        <span class="token keyword">return</span> np<span class="token punctuation">.</span>dot<span class="token punctuation">(</span>X<span class="token punctuation">,</span> self<span class="token punctuation">.</span>w_<span class="token punctuation">[</span><span class="token number">1</span><span class="token punctuation">:</span><span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token operator">+</span> self<span class="token punctuation">.</span>w_<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span>

    <span class="token keyword">def</span> <span class="token function">predict</span><span class="token punctuation">(</span>self<span class="token punctuation">,</span> X<span class="token punctuation">)</span><span class="token punctuation">:</span>
        <span class="token triple-quoted-string string">&quot;&quot;&quot;
        预测类别变量，只返回 1 或者 -1
        :param X:
        :return:
        &quot;&quot;&quot;</span>
        <span class="token keyword">return</span> np<span class="token punctuation">.</span>where<span class="token punctuation">(</span>self<span class="token punctuation">.</span>net_input<span class="token punctuation">(</span>X<span class="token punctuation">)</span> <span class="token operator">&gt;=</span> <span class="token number">0.0</span><span class="token punctuation">,</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">)</span>
</code></pre> <div class="line-numbers-wrapper"><span class="line-number">1</span><br><span class="line-number">2</span><br><span class="line-number">3</span><br><span class="line-number">4</span><br><span class="line-number">5</span><br><span class="line-number">6</span><br><span class="line-number">7</span><br><span class="line-number">8</span><br><span class="line-number">9</span><br><span class="line-number">10</span><br><span class="line-number">11</span><br><span class="line-number">12</span><br><span class="line-number">13</span><br><span class="line-number">14</span><br><span class="line-number">15</span><br><span class="line-number">16</span><br><span class="line-number">17</span><br><span class="line-number">18</span><br><span class="line-number">19</span><br><span class="line-number">20</span><br><span class="line-number">21</span><br><span class="line-number">22</span><br><span class="line-number">23</span><br><span class="line-number">24</span><br><span class="line-number">25</span><br><span class="line-number">26</span><br><span class="line-number">27</span><br><span class="line-number">28</span><br><span class="line-number">29</span><br><span class="line-number">30</span><br><span class="line-number">31</span><br><span class="line-number">32</span><br><span class="line-number">33</span><br><span class="line-number">34</span><br><span class="line-number">35</span><br><span class="line-number">36</span><br><span class="line-number">37</span><br><span class="line-number">38</span><br><span class="line-number">39</span><br><span class="line-number">40</span><br><span class="line-number">41</span><br><span class="line-number">42</span><br><span class="line-number">43</span><br><span class="line-number">44</span><br><span class="line-number">45</span><br><span class="line-number">46</span><br><span class="line-number">47</span><br><span class="line-number">48</span><br><span class="line-number">49</span><br><span class="line-number">50</span><br><span class="line-number">51</span><br><span class="line-number">52</span><br><span class="line-number">53</span><br><span class="line-number">54</span><br><span class="line-number">55</span><br><span class="line-number">56</span><br><span class="line-number">57</span><br><span class="line-number">58</span><br><span class="line-number">59</span><br><span class="line-number">60</span><br><span class="line-number">61</span><br><span class="line-number">62</span><br><span class="line-number">63</span><br><span class="line-number">64</span><br><span class="line-number">65</span><br><span class="line-number">66</span><br><span class="line-number">67</span><br><span class="line-number">68</span><br><span class="line-number">69</span><br><span class="line-number">70</span><br><span class="line-number">71</span><br><span class="line-number">72</span><br><span class="line-number">73</span><br><span class="line-number">74</span><br></div></div><h2 id="示例"><a href="#示例" class="header-anchor">#</a> 示例</h2> <h3 id="例1-鸢尾花数据集可视化"><a href="#例1-鸢尾花数据集可视化" class="header-anchor">#</a> 例1：鸢尾花数据集可视化</h3> <iframe src="https://nbviewer.jupyter.org/github/liweiwei1419/Machine-Learning-is-Fun/blob/master/Perceptron-learning/notebook/%E9%B8%A2%E5%B0%BE%E8%8A%B1%E6%95%B0%E6%8D%AE%E9%9B%86%E5%8F%AF%E8%A7%86%E5%8C%96.ipynb" width="800" height="1000"></iframe> <h3 id="例2-感知机算法的-python-实现"><a href="#例2-感知机算法的-python-实现" class="header-anchor">#</a> 例2：感知机算法的 Python 实现</h3> <iframe src="https://nbviewer.jupyter.org/github/liweiwei1419/Machine-Learning-is-Fun/blob/master/Perceptron-learning/notebook/%E6%84%9F%E7%9F%A5%E6%9C%BA%E7%AE%97%E6%B3%95%E7%9A%84%20Python%20%E5%AE%9E%E7%8E%B0.ipynb" width="800" height="1000"></iframe> <h2 id="参考资料"><a href="#参考资料" class="header-anchor">#</a> 参考资料</h2> <p>[1] 李航. 统计学习方法（第 2 版）第 2 章“感知机”. 北京：清华大学出版社，2019.</p> <p>[2] 刘建平. <a href="http://www.cnblogs.com/pinard/p/6042320.html" target="_blank" rel="noopener noreferrer">感知机原理小结<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>、<a href="http://www.cnblogs.com/pinard/p/5970503.html" target="_blank" rel="noopener noreferrer">梯度下降（Gradient Descent）小结<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>.</p> <p>[3] 码农场. <a href="http://www.hankcs.com/ml/the-perceptron.html" target="_blank" rel="noopener noreferrer">感知机的学习<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>.</p> <p>[4] 知乎网友. <a href="https://www.zhihu.com/question/26526858" target="_blank" rel="noopener noreferrer">如何理解感知机学习算法的对偶形式？<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>.</p> <p>说明：对偶形式把累加变成了乘法运算，并且引入了 Gram 矩阵。</p> <p>[5] Sebastian Raschka. Python 机器学习 第 2 章. 北京：机械工业出版社，2019.</p> <p>（本节完）</p></div> <footer class="page-edit"><!----> <div class="last-updated"><span class="prefix">上次更新:</span> <span class="time">4/10/2021, 6:19:58 PM</span></div></footer> <!----> </main></div><div class="global-ui"><!----></div></div>
    <script src="/book/assets/js/app.a4355b0d.js" defer></script><script src="/book/assets/js/2.ba785ee6.js" defer></script><script src="/book/assets/js/46.33bc2083.js" defer></script>
  </body>
</html>
